Fast Polyhedral Adaptive Conjoint Estimation

We propose and test a new adaptive conjoint analysis method that draws on recent polyhedral “interior-point” developments in mathematical programming. The method is designed to offer accurate estimates after relatively few questions in problems involving many parameters. Each respondent’s ques-tions are adapted based upon prior answers by that respondent. The method requires computer support but can operate in both Internet and off-line environments with no noticeable delay between questions. We use Monte Carlo simulations to compare the performance of the method against a broad array of relevant benchmarks. While no method dominates in all situations, polyhedral algorithms appear to hold significant potential when (a) metric profile comparisons are more accurate than the self-explicated importance measures used in benchmark methods, (b) when respondent wear out is a concern, and (c) when product development and/or marketing teams wish to screen many features quickly. We also test hybrid methods that combine polyhedral algorithms with existing conjoint analysis methods. We close with suggestions on how polyhedral methods can be used to address other marketing problems.Sloan School of Management and the Center for Innovation in Product Development at MI

Application and Test of Web-based Adaptive Polyhedral Conjoint Analysis

In response to the need for more rapid and iterative feedback on customer preferences, researchers are developing new web-based conjoint analysis methods that adapt the design of conjoint questions based on a respondent’s answers to previous questions. Adapting within a respondent is a difficult dy-namic optimization problem and until recently adaptive conjoint analysis (ACA) was the dominant method available for addressing this adaptation. In this paper we apply and test a new polyhedral method that uses “interior-point” math programming techniques. This method is benchmarked against both ACA and an efficient non-adaptive design (Fixed). Over 300 respondents were randomly assigned to different experimental conditions and were asked to complete a web-based conjoint exercise. The conditions varied based on the design of the con-joint exercise. Respondents in one group completed a conjoint exercise designed using the ACA method, respondents in another group completed an exercise designed using the Fixed method, and the remaining respondents completed an exercise designed using the polyhedral method. Following the conjoint exer-cise respondents were given $100 and allowed to make a purchase from a Pareto choice set of five new-to-the-market laptop computer bags. The respondents received their chosen bag together with the differ-ence in cash between the price of their chosen bag and the$100. We compare the methods on both internal and external validity. Internal validity is evaluated by comparing how well the different conjoint methods predict several holdout conjoint questions. External validity is evaluated by comparing how well the conjoint methods predict the respondents’ selections from the choice sets of five bags. The results reveal a remarkable level of consistency across the two validation tasks. The polyhe-dral method was consistently more accurate than both the ACA and Fixed methods. However, even better performance was achieved by combining (post hoc) different components of each method to create a range of hybrid methods. Additional analyses evaluate the robustness of the predictions and explore al-ternative estimation methods such as Hierarchical Bayes. At the time of the test, the bags were proto-types. Based, in part, on the results of this study these bags are now commercially available.The Sloan School of Management, the Center for Innovation in Product Development at MIT and the EBusiness Center at MI

Understanding Consumer Preferences for Explanations Generated by XAI Algorithms

Explaining firm decisions made by algorithms in customer-facing applications is increasingly required by regulators and expected by customers. While the emerging field of Explainable Artificial Intelligence (XAI) has mainly focused on developing algorithms that generate such explanations, there has not yet been sufficient consideration of customers' preferences for various types and formats of explanations. We discuss theoretically and study empirically people's preferences for explanations of algorithmic decisions. We focus on three main attributes that describe automatically-generated explanations from existing XAI algorithms (format, complexity, and specificity), and capture differences across contexts (online targeted advertising vs. loan applications) as well as heterogeneity in users' cognitive styles. Despite their popularity among academics, we find that counterfactual explanations are not popular among users, unless they follow a negative outcome (e.g., loan application was denied). We also find that users are willing to tolerate some complexity in explanations. Finally, our results suggest that preferences for specific (vs. more abstract) explanations are related to the level at which the decision is construed by the user, and to the deliberateness of the user's cognitive style.Comment: 18 pages, 1 appendix, 3 figures, 4 table

Measuring Consumer Preferences Using Conjoint Poker

We develop and test an incentive-compatible Conjoint Poker (CP) game. The preference data collected in the context of this game are comparable to incentive-compatible choice-based conjoint (CBC) analysis data. We develop a statistical efficiency measure and an algorithm to construct efficient CP designs. We compare incentive-compatible CP to incentive-compatible CBC in a series of three experiments (one online study and two eye-tracking studies). Our results suggest that CP induces respondents to consider more of the profile-related information presented to them compared with CBC

Fast Polyhedral Adaptive Conjoint Estimation

We propose and test new "polyhedral" question design and estimation methods that use recent developments in mathematical programming. The methods are designed to offer accurate estimates after relatively few questions in problems involving many parameters. With polyhedral question design, each respondent's questions are adapted based upon prior answers by that respondent to reduce a feasible set of parameters as rapidly as possible. Polyhedral estimation provides estimates based on a centrality criterion (the "analytic center" of the feasible parameter set). The methods require computer support but can operate in both Internet and other computer-aided environments with no noticeable delay between questions. We evaluate the proposed methods using two approaches. First, we use Monte Carlo simulations to compare the methods against established benchmarks in a variety of domains. In the simulations we compare polyhedral question design to three benchmarks: random selection, efficient Fixed designs, and Adaptive Conjoint Analysis (ACA). We compare polyhedral estimation to Hierarchical Bayes estimation for each question design method. The simulations evaluate the methods across different levels of respondent heterogeneity, response accuracy, and numbers of questions. For low numbers of questions, polyhedral question design does best (or is tied for best) for all domains. For high numbers of questions, efficient Fixed designs do better in some domains. The best estimation method depends on respondent heterogeneity and response accuracy. Polyhedral (analytic center) estimation shows particular promise for high heterogeneity and/or for low response errors. The second evaluation employs a large-scale field test. The field test involved 330 respondents, who were randomly assigned to a question-design method and asked to complete a web-based conjoint exercise. Following the conjoint exercise, respondents were given $100 and allowed to make a purchase from a Pareto choice set of five new-to-the-market laptop computer bags. The respondents received their chosen bag together with the difference in cash between the price of their chosen bag and the$100. We compare the question-design and estimation methods on both internal validity (holdout tasks) and external validity (actual choice of a laptop bag). The field test findings are consistent with the simulation results and offer strong support for the polyhedral question design method. The preferred estimation method varied based on the question design method, although Hierarchical Bayes estimation consistently per-formed well in this domain. The findings reveal a remarkable level of consistency across the validation tasks. They suggest that the proposed methods are sufficiently promising to justify further development. At the time of the test, the bags were prototypes. Based, in part, on the results of this study the bags were launched successfully and are now commercially available. Sales of the features of the laptop bags were consistent with conjoint-analysis predictions

New approaches to idea generation and consumer input in the product development process

Thesis (Ph. D.)--Massachusetts Institute of Technology, Sloan School of Management, 2004."June 2004."Includes bibliographical references.This thesis consists of five related essays which explore new approaches to help design successful and profitable new products. The primary focus is the front end of the process where the product development team is seeking improved input from customers and improved ideas for developing products based on that input. Essay 1 examines whether carefully tailored ideation incentives can improve creative output. The influence of incentives on idea generation is studied using a formal model of the ideation process. A practical, web-based, asynchronous "ideation game" is developed, allowing the implementation and test of various incentive schemes. Using this system, an experiment is run, which demonstrates that incentives do have the capability to improve idea generation, confirms the prediction from the theoretical analysis, and provides additional insight on the mechanisms of ideation. Essay 2 proposes and tests new adaptive question design and estimation algorithms for partial-profile conjoint analysis. The methods are based on the identification and characterization of the set of parameters consistent with a respondent's answers . This feasible set is a polyhedron defined by equality constraints, each paired-comparison question yielding a new constraint. Polyhedral question design attempts to reduce the feasible set of parameters as rapidly as possible. Analytic Center estimation relies on the center of the feasible set. The proposed methods are evaluated relative to established benchmarks using simulations, as well as a field test with 330 respondents. Essay 3 introduces polyhedral methods choice-based conjoint analysis, and generalizes the concept of D-efficiency to individual adaptation. The performance of the methods is evaluated(cont.) using simulations, and an empirical application to the design of executive education programs is described. Essay 4 generalizes the existing polyhedral methods for adaptive choice-based conjoint analysis by taking response error into account in the adaptive design and estimation of choice-based polyhedral questionnaires. The validity of the proposed approach is tested using simulations. Essay 5 studies the impact of Utility Balance on efficiency and bias. A new definition of efficiency (M-efficiency) is also introduced, which recognizes the necessity to match preference questions with the quantities used in the ultimate managerial decisions.by Olivier Toubia.Ph.D

1 Intrinsic versus Image-Related Utility in Social Media: Why Do People Contribute Content to Twitter?

We empirically study the motivations of users to contribute content to social media in the context of the popular microblogging site Twitter. We focus on non-commercial users who do not benefit financially from their contributions. Previous literature suggests two main sources of utility that may motivate these users to post content: intrinsic utility and image-related utility. We leverage the fact that these two types of utility give rise to different predictions as to whether users should increase their contributions when their number of followers increases. To address the issue that the number of followers is endogenous, we conducted a field experiment in which we exogenously added followers (or follow requests in the case of protected accounts) to a set of users over a period of time, and compared their posting activities to those of a control group. We estimated each treated user’s utility function using a dynamic discrete choice model. While our results are consistent with both types of utility being at play, our model suggests that imagerelated utility is larger for most users. We discuss the implications of our findings for the evolution of Twitter and the type of value firms may derive from such platforms in the future